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Purpose: To assess how the use of games contributes to students’ science learning, interests, and attitudes 
about science. 

Methodology: The study sample was middle and high-school students in a large urban school district. A 
total of 1191 students participated in the game. The majority of students were Hispanic females of low 
socio-economic backgrounds. Students were recruited by science teachers at their schools. A mixed- 
method, pre-posttest design was used to measure students’ science knowledge, attitudes, and interests. The 
instruments were piloted in the previous year with a comparable student group, and were found to be 
reliable measures. The instruments were distributed, using a web-based format, to students in their science 
classes. Items on the science test were developed based on state standards. Students were offered incentives 
to participate in the game. 

Results: Findings from a paired sample of 391 students revealed a statistically significant increase in the 
number of science test items answered correctly from pre- to posttests. Eta squared results indicated a 
moderate effect size. There was also a statistically significant increase in the overall mean interest rating, 
but a decrease in the mean attitude rating over the study period. Pearson’s r revealed a strong, positive 
correlation between students’ interests and attitudes about science. 

Conclusions: There was evidence that, as students’ interests in science increased, their attitudes about 
science increased. Moreover, as students’ pre- interests and attitudes about science increased, their post 
science assessment scores increased. 

Recommendations: There were limitations to the study, including the lack of a comparison group who did 
not participate in the game. Nevertheless, the findings suggest immediate benefits in the use of games to 
improve middle and high-school students’ science performance. Future studies might investigate the long- 
term impact of the game on students’ science achievement and career pathways. 

Additional data: (There are 7 tables and graphs included in the paper.) 
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Chevron expanded its funding of the New Digital Energy game (NDE) to cover a three-year period (2010— 
2013). This collaboration with Tietronix Software, Inc. and a large school district in Houston, Texas led to 
the development of a computer game designed to transform the learning experience of middle and high- 
school students in science. NDE combines aspects of strategy, construction, and game management, 
requiring players to build energy companies, gain dominant market share and meet the needs of cities 
throughout the United States. Cooperative teams of students play against artificial intelligence, competing 
across three levels of difficulty. Variations in difficulty are incorporated into lessons that students must 
master in order to open options within the game. Lessons and questions are designed to lead players to 
game-play decisions that require understanding of physics, chemistry, earth science, and math concepts. A 
meta-site is available outside of game play for students to supplement their learning and success for 
subsequent game play. Lessons are crafted within the game to encourage learning through inquiry. Lessons 
are aligned to the Texas Essential Knowledge and Skills standards (TEKS) (NDE, correspondence, 2011). 
The program included field trips to the Ocean Star Offshore Drilling Rig & Museum in Galveston and the 
Houston Museum of Natural Sciences’ WEISS Energy Hall. Students were awarded prizes for high scores 
and participation in the game. 


Study Sample and Methods 


All middle and high school science teachers were 
invited to have their students participate in the 
NDE game. Students played the game during the 
fall semester for approximately 3 months. Figure 1 
shows a 29.1 percent increase in the number of 
students who registered for the game and a 45.1 
percent increase in the number of students who 
played the game over the past two years. Sixteen 
schools and 26 teachers were represented in the 
2011-2012 data among 85 middle and high schools 
invited to participate. 

Students’ interests and attitudes concerning 
science were measured using a web-based pre- and 
post survey. Tietronix Software, Inc. worked 
collaboratively with the Secondary Curriculum, 
Instruction, and Assessment Department to develop 
the instrument. Students’ knowledge of science 
concepts were also measured using a pre -post 
assessment design. Students were provided 
electronic links to the instruments on the game 


Web site and through their science teachers. While 
the level of participation in the game could have 
influenced students’ interests, attitudes, and science 
energy knowledge, participation levels were not 
captured by Tietronix Software, Inc. to maintain 
students’ anonymity and confidentiality. 



Numberof Numberof 

students students 

registered playing 


Figure 1. Number of all students who registered and 
participated in the NDE in game, past two academic 
years. 
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G/T Eco. Spec. Ed. Male Female Afri. Am. Asian Hisp. White Other 

Disadv. 


Figure 2. Demographic characteristics of NDE game matched sample, 2011-2012 
Note: Sample size - 391 


Results of a matched-paired sample of 391 
students who completed the four measures (pre- and 
post science energy knowledge assessments as well 
as the pre- and post interest/attitude surveys) were 
extracted from the data for analysis. Students in the 
matched sample were enrolled at seven middle and 
two high schools. Figure 2 presents demographic 
characteristics of the matched student sample. The 
majority of students were economically 
disadvantaged (84.1 percent), male (61.9 percent), 
and Hispanic (62.3 percent). In addition, 43.7 
percent of the sample was enrolled in gifted and 
talented programs (G/T), while 2.8 percent received 
special education services. Representation of 
Hispanic students in the sample was comparable to 
the districtwide Hispanic student population in the 
current year (62.4 percent). African American 
students were overrepresented in the NDE sample 
compared to the district wide percentage (32.7 vs. 
25.1 percent). G/T and special education students 
were also more prevalent in the district than in the 
study sample (15.1 and 7.8 percent respectively) 
(PEIMS, 2011-2012). 

Descriptive statistics, including means and 
standard deviations, were calculated based on 
student responses using IBM SPSS software. Paired 
sample data measuring the percent change in 
students’ raw scores on the knowledge-based 
science energy assessment and the percent change in 
students’ level of agreement (strongly agree or 
agree) with survey items measuring interest and 
attitudes were reported. Pearson correlation 
coefficient measured the association between pre- 


post science energy knowledge and pre- and post 
interest/attitudes (p<.05). 


Background 


Over the past decade, educators have sought 
innovative approaches to strengthen student 
achievement in science. The National Science 
Foundation (2008) found that computer games 
enhance the educational experience of students, 
partly due to the multifaceted process required to 
learn science (Honey and Hilton, 2010). Mundie 
(2011) maintains that “Technology has the potential 
to help reinvent the education process, and excite 
and inspire young learners to embrace science, math 
and technology” (p.l). In efforts to add to the body 
of knowledge, researchers have discovered that, “the 
best educational activities connect not only to 
important concepts but also to students’ interests 
and passions” (Games for Learning Institute, 2011, 
p.l). 

Computer games are inherently interactive, and 
have, increasingly, become a new and innovative 
approach to teaching and learning science as they 
capture the attention of youth. In addition, computer 
games allow students to be transported into another 
reality and engage in activities that simulate the real 
world while exploring natural phenomena that 
cannot be directly observed (Games Research, 2011; 
Mundie, 2011). 
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Table 1: Change in Raw Scores of Students on Science Assessment on Pretest Compared to Posttest, 2011-2012 



Positive 

Change 

No Change [ 

1 Negative Change | 

Total 

Grade 

n 

% 

n 

% 

n 

% 

N 

6 

84 

52.2t 

22 

13.7 

55 

34.2 

161 

7 

24 

39.3 

10 

16.4 

27 

44.3 

61 

8 

42 

43.7 

17 

17.7 

37 

38.5 

96 

9 

14 

42.4 

4 

12.1 

15 

45.5 

33 

10 

12 

48.0 

4 

16.0 

9 

36.0 

25 

11 

5 

55.6t 

1 

11.1 

3 

33.3 

9 

12 th 

2 

83.3t 

2 

33.3 

2 

33.3 

6 

All Grades 

183 

46.8 

60 

15.3 

148 

37.9 

391 


Note: | Represents the majority positive change at specified grade level 


Research was conducted in HISD on the first year 
of implementation of the NDE program. The study 
found an increase in students’ interest and attitudes 
in science (Holmes, 2011). While there continue to 
be gaps in the research on which gaming features 
contribute to student learning, this study builds on 
the body of evidence relative to the effectiveness of 
games toward enhancing students’ interest and 
attitudes about science and their ability to leant 
science concepts through games. 


What was the impact of the NDE game on 
students’ acquisition of science energy 
knowledge? 


Table 1 presents the number and percent change 
in the raw scores of students on an assessment that 
measured science energy knowledge. Results reflect 
positive change, no change, and negative change 
scores by grade level between pre- and post 
assessments. The overall percentage of students with 
positive change scores was 46.8 percent compared 
to 15.3 percent of students with no change in their 
assessment scores. The percentage of students with 
positive change scores ranged from 39.3 percent to 
83.3 percent at various grade levels. Table 1 also 
shows that the majority of 6 th , 11 th , and 12 th -grade 
students showed positive changes in their test 
scores. 

A paired t-test was conducted based on the 
percent of items that students answered correct on 
the pre- and post science energy assessment. The 
analysis showed a statistically significant increase in 
the percent of items correct from pretest (M=40.35, 
SD=19.8) to posttest (M=45.40 and SD=22.01), 
t(390)=4.35, p=.000 (two-tailed). The mean 

difference was 5.05. The 95% confidence interval 
ranged from 2.765 to 7.338. The eta squared statistic 
(.05) indicated a moderate effect size of the 
program, based on science assessment performance. 


To what extent was students’ interest in science 
influenced by game participation? 


A Likert-type scale was used to rate the frequency 
that students engaged in activities conceptualized as 
measuring interest in science before and after 
participating in the game. The scale used in the 
survey was: very often-4; regularly-3; sometimes-2; 
and never or hardly ever-1. The results of 391 
students (33% response rate for the paired sample) 
are included in the analysis. Ratings of very often 
and regularly were combined to show the percent 
change from pre- to post survey. The results are 
shown in Figure 3 (page 4). 

It is apparent in Figure 3 that the highest percent 
change in students responding very often or 
regularly to interest-related items was participation 
in science competitions (112.2%) and talking to 
parents or family members about science (95.4%). 
The percent of students indicated very often or 
regularly from pre- to post survey increased by more 
than 50 percent on the following items: talked to 
their friends about science (92.3 percent), attended 
science groups (78.4 percent), and participated in 
after-school science activities (65.9 percent). 

A paired t-test analysis showed a statistically 
significant increase in the overall mean interest 
rating from pre survey (M=2.27, SD=.67) to post 
survey (M=2.37, SD=.73, t(390)=3.41, pc.OOl). The 
95% Cl [.041, .241] provides additional evidence of 
a significant difference in how students rated 
frequency of participation on items related to 
interest in science before and after the game. The eta 
squared statistic (.03) indicated a small effect of the 
program on students’ interest in science. 


To what extent were students’ attitudes about 
science influenced by the NDE game? 


Survey items measuring students’ attitudes about 
science were based on the following categories: 
strongly agree-4, agree-3, disagree -2, and strongly 
disagree- 1. The same 391 students whose results 
were used to measure their interest were also used to 
measure their attitudes about science from pre to 
post survey. The findings depicting the combined 
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Student Interest 


Had fun whenlearningscience. 
Talked to my teacher about science. 
Participated in science competitions. 
Talked to my friends about science. 
Attended science study groups. 
Visited websites about science. 
Participated in after-school science activities. 
Talked about science to parents or family members. 
Watched news on TV that involved science concepts. 
Read books or magazines about science outside of school. 



B Very Often/Regularly 


Figure 3. Percent change in student’s ratings of very often or regularly on survey items measuring interest in 
science energy from pre to post assessment. 

Note: Sample size =391 


categories of strongly agree and agree, are presented 
in Figure 4 (page 5). 

The highest percent change in students indicating 
strongly agree or agree to attitude-related items were 
that they like to study science (49.7 percent), are 
interested in learning more science (46.0 percent), 
and would like to have a career involving energy 
(oil, wind, solar, nuclear, biomass, and natural gas) 
(45.8 percent). Students were least likely to change 
that they strongly agreed or agreed the game would 
help them learn difficult science theories that they 
may not understand without seeing the game (0.0 
percent) and studying science is worth it because it 
will help them get into college (3.7 percent). 

A paired t-test analysis showed a statistically 
significant decrease in the overall mean rating from 
pre survey (M=2.91, SD=.76) to the post survey 
(M=2.82, SD=.77, t(390)=-2.85, p<.005). The 95% 
Cl was [-.146, -.027], The eta squared statistic (.02) 
indicated a small effect on the student samples’ 
attitudes about science after the game. 


Was there an association between students’ 
science energy knowledge, interest, and attitudes 
regarding science? 


The relationship between students’ pre- and post 
science energy knowledge, interest and attitudes 
about science were investigated using the Pearson 
product-moment correlation coefficient (Table 2) 
(page 5). Preliminary analysis was conducted to 
ensure no violation of test assumptions. Some of the 
most notable results were the strong positive 
associations between students’ interest and attitudes 
[r=.678, n=391, p<.01]. Specifically, as students’ 
interest in science increased, their attitudes about 
science increased. The results were highly 
statistically significant. The data also showed as pre 
interest and pre attitudes scores increased post 
assessment scores increased [r=. 108, n=391, 

p=.034] and [r=. 192, n=391, p=.000], respectively. 


What were students’ overall impressions of the 
NDE game? 
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Student Attitudes 


I would like to work on science projects as an adult. 

I would like to haveacareerinvolvingenergy 
(oil. wind, solar, nuclear, biomass, natural gas). 

I want to study science after high school. 
I would like to haveacareerinvolvingscience. 


I study science becausel know it is useful to me. 

Making an effort to study science is worth it because it will 
help me get into college. 

Studying science is worthwhile because it will improve my 
career opportunities. 

Science energy game will help me leam difficult science 
theories that I may not understand without seeing in the game. 

I like studying science. 


Science will help me become successful. 


I am interested in learningmore science. 


■ Strongly Agree/Agree 



Figure 4. Percent change in student’s ratings of strongly agree or agree on survey items measuring attitudes about 
science energy from pre to post assessment. 

Note: Sample size =391 


Students were asked to provide additional 
comments about the NDE game following 
participation. The open-ended responses of all 
students who completed the post interest /attitudes 
survey were included in the analysis. Responses 
were analyzed using IBM SPSS Text Analytics and 
the SurveyMonkey Text Analysis feature in order to 
triangulate emerging themes. Themes are based on 
comments of 158 respondents. 

The most prevalent themes focused on students’ 
positive impressions of the game (n=69). Students 
describe the game as “fun” (n=38), “awesome” 


(n=9), “interesting” (n=9), “great” (n=5), “cool” 
(n=4), and “amazing” (n=2). Some students 
mentioned that the game was “helpful” (n=15) and a 
learning experience. One student wrote, "The game 
was very fun and helped me learn about different 
types of resources and which are best. ” Another 
student wrote, “7 like it because it is educational for 
me to learn more things about science in the world 
or even everywhere I go. It’s a fun game with a[n] 
educational twist that I would like to learn more 
stuff about. ” 


Table 2: Pearson Product-Moment Correlations Between Measures of Achievement, Interest and Attitude 


Measures 1 2 3 4 5 
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Pre Science Assessment (1) 

1.00 





Post Science Assessment (2) 

.399** 

1.00 




Pre Interest (3) 

.141** 

* 

oo 

o 

1.00 



Post Interest (4) 

.005 

.005 

.679** 

1.00 


Pre Attitude (5) 

.232** 

192** 

.678** 

.508** 

1.00 

Post Attitude (6) 

.110* 

.169** 

.589** 

.718** 

.695** 


**Correlation is significant at the 0.01 level (2-tailed) 
Correlation is significant at the .05 level (2-tailed) 
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Table 3: Teacher Perceptions Regarding the NDE Game’s Impact on Instruction, 201 1-2012 


Since participating in the game, indicate the level of agreement that you 
changed your instructional practices in the following areas: 

Strongly 

Agree 

% 

Agree 

% 

Disagree 

% 

Strongly 

Disagree 

% 

n 

Assign more web-based instructional activities 

25.0 

50.0 

25.0 

0.0 

8 

Assign more team projects and cooperative group tasks 

14.3 

57.1 

28.6 

0.0 

7 

Access more websites about science concepts 

12.5 

75.0 

12.5 

0.0 

8 

The science game exposed my students to information that required the 
use of critical thinking skills. 

50.0 

50.0 

0.0 

0.0 

8 

Students seemed more interested in science after participating in the 
game. 

37.5 

62.5 

0.0 

0.0 

8 

I adjusted my instructional practices based on students’ inquiry and 
feedback. 

25.0 

50.0 

25.0 

0.0 

8 

I used additional resources to support the concepts covered in the game. 

37.9 

50.0 

12.5 

0.0 

8 

Students seemed more motivated to do well in science after 
participating in the game. 

25.0 

75.0 

0.0 

0.0 

8 


Several students perceived long-term benefits of 
participating in the game. Some comments were: “It 
is a great experience that will help me have a back 
up in my career. It will help me understand and 
become successful. ” Another student wrote: “This 
game has not only taught me new things about 
science but also taught me new things about running 
a business. ” Students’ general satisfaction was 
expressed by replying “I like if ’ (n=26) and 
“ thanks ” for allowing them to participate (n=5). 

Three students noted technical problems with the 
game. One student commented: “The game was very 
interesting, I would like to play every year. But, 
there are a lot of problems that need to be fixed. ” 
Suggestions included resolving the “ glitches ” and 
the game “freezing.” 


What were NDE teachers’ perceptions regarding 
the game’s impact on student performance? 


Of the 26 participating teachers, eleven middle 
and high-school teachers who had at least five 
students complete the pre- and post surveys and 
assessments were asked to provide input on the 
program’s impact on their instructional practices. 
Eight teachers provided feedback. Initial 
background information on training, professional 
development, and students’ access to the game was 
captured. Teacher perceptions are presented in 
Table 3. 

The majority of teachers indicated that they 
received adequate training from the game 
administrators and had adequate technology skills to 
assist students in playing the game. On the other 
hand, two teachers revealed that they were unable to 
access technical support to resolve computer-related 
issues in a timely manner. 

Relative to instruction, the majority of teachers 
sought professional development to increase their 
understanding of concepts covered in the game 
(87.5 percent) and 75.0 percent agreed to adjusting 


their instructional practices based on students’ 
inquiry and feedback. The highest percentage of 
teachers expressed strong agreement that the science 
game exposed their students to information that 
required the use of critical thinking skills (50 
percent). All respondents noted agreement with the 
statement that students’ interest and motivation to 
do well in science increased following participation 
in the game. The most disagreement among teachers 
concerned changing their instructional practices 
since participating in the game was assigning more 
team projects and assigning more cooperative group 
tasks (28.6 percent). 


Discussion 


Chevron’s partnership and financial support of the 
NDE game has resulted in the game’s expansion 
from 2010 to 2013. The game was designed to 
create a learning environment that allows students to 
work together to solve problems that are aligned to 
their learning levels and to science instructional 
content. 

Major objectives of the program are to increase 
students’ interest and attitudes about science as well 
as increase their knowledge of science energy 
concepts. Following the first year of the program, 
survey results showed that students became more 
involved in science-related activities during and 
outside of school. There was also evidence that 
students increased their aspirations to pursue science 
careers and careers involving energy as adults. 
Outcomes noted in the second year revealed a large 
proportion of students indicated significant positive 
change in the extent that they participated in science 
competitions, talked to others about science, and 
read about science outside of school after 
participating in the game. Students also expressed 
the impact of the game on their interest in pursuing 
careers in science. At the same time, there was a 
significant decrease in the overall rating on items 
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that measured students’ attitude about science. 
However, positive changes in attitude were noted in 
areas including that they liked to study science and 
had interest in learning more science from pre- to 
post survey. 

The game’s impact on students and teachers was 
also noted in qualitative responses provided by 
teachers. Specifically, teachers observed an increase 
in students’ interest and motivation to do well in 
science after participating in the game. The majority 
of teachers modified their instructional practices to 
incorporate more team and web-based instructional 
projects. The game also served as motivation for 
teachers to seek additional professional development 
to increase their understanding of science energy 
concepts. Considering all of these factors, the 
research provides promising evidence for using 
computer games to improve student learning in 
science. 

There were several limitations in the study, 
including the lack of a comparison group and 
posttest data on the majority of students. 
Specifically, the number of students who completed 
both pre- and post surveys compared to the number 
of students who registered to play the game was 
substantially lower (391 out of 1191 students). The 
evaluation relied on teachers to encourage their 
students to complete the instruments, which may 
have contributed to missing data and data to include 
for post survey analysis. Moreover, it was evident 
that teachers from all middle and high schools in the 
district were not represented in the data, thus 
students in all of the targeted schools did not 
participate. Future studies should consider having 
students and teachers sign written agreement to 
inform them about evaluation expectations and 
indicate their commitment to the evaluation. 
Although incentives were offered, these incentives 
should be more closely tied to study completion. In 
addition, efforts to recruit more teachers in all 
targeted schools should be strengthened to ensure a 
more representative study sample. 

With Chevron’s support, recommendations are 
to continue game development in similar student 
populations, and at other grade levels and content, to 
identify strategies that promote academic success 
for students who struggle learning science concepts. 
In addition, further enhancements of the game may 
help students learn difficult concepts and theories. 
Monitoring students’ perceptions will support the 
long-term impact of the NDE game experience and 
the sustainability of interest and attitudes about 
science over time. 
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